• Updating the AOS and AIS data is handled separately in neon-aqu-aos.
  • AOP flight times are determined from L1 radiance metadata.

Summary for PRLA

  • Green flightlines from multiple dates in 2019 (July 26, 27) and 2020 (June 24, July 2)
  • In 2019, chl sampling from July 11 (2 weeks before), chemistry August 6 (11 days after), sensor data available
  • In 2020, chl sampling from June 15 (9 days before) and the following sample was very high. chemistry data not all available from closest sampling date (only TOC not DOC). sensor data available for first flight date but not later date (which had a higher spectra with more pronounced peaks)
  • Also AOP data collected in 2016 and 2017 under yellow conditions but no ground-based data
mysite = 'PRLA'

Chlorophyll

AOS Sampling

  • From processed chl a files on Box
  • reported in ug/L
  • seasonal sampling, 2 analytes

See how many locations chl data is collected from

See how many locations chl data is collected from (variable Y axis)

Availability of chlorophyll AOS data for each flight date

  • Default threshold 1 = 30 days
  • Default threshold 2 = 60 days
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2019-07-11 NA 1107 1107 FALSE FALSE
2017-06-21 NA 2019-07-11 NA 750 750 FALSE FALSE
2017-06-26 NA 2019-07-11 NA 745 745 FALSE FALSE
2019-07-26 2019-07-11 2019-09-10 15 46 15 TRUE TRUE
2019-07-27 2019-07-11 2019-09-10 16 45 16 TRUE TRUE
2019-07-30 2019-07-11 2019-09-10 19 42 19 TRUE TRUE
2020-06-24 2020-06-15 2020-09-29 9 97 9 TRUE TRUE
2020-06-26 2020-06-15 2020-09-29 11 95 11 TRUE TRUE
2020-07-02 2020-06-15 2020-09-29 17 89 17 TRUE TRUE
flightdate aos_match days
2019-07-26 2019-07-11 15
2019-07-27 2019-07-11 16
2019-07-30 2019-07-11 19
2020-06-24 2020-06-15 9
2020-06-26 2020-06-15 11
2020-07-02 2020-06-15 17
  • Make sure plant algae lab units are all micrograms per liter
  • Check sample conditions
## [1] "microgramsPerLiter"
## [1] "condition ok"

chl AOS x flightdates

These are the closest chl AOS values for each flight date

aos_match flightdate days pheophytin chlorophyll a
2019-07-11 2019-07-26 15 3.01 2.31
2019-07-11 2019-07-27 16 3.01 2.31
2019-07-11 2019-07-30 19 3.01 2.31
2020-06-15 2020-06-24 9 4.20 3.45
2020-06-15 2020-06-26 11 4.20 3.45
2020-06-15 2020-07-02 17 4.20 3.45

Algal Community (TODO)

TO DO!!!!!

  • Relative abundance by division, class for diatom slides and soft algae count
  • Top 10 morphospecies for each method?

AIS Sensor

The EXO total algae sensor is a dual‐channel fluorometer that uses a 470nm excitation beam that excites chlorophyll a and a second 590 nm excitation beam that excites the phyocyanin accessory pigment found in blue‐green algae (cyanobacteria). Chlorophyll concentration is a biogeochemically relavant parameter that is readily available by remote sensing and can be can serve as a proxy for phytoplankton biomass and light attenuation (Oestreich et al., 2016, Ganju et al., 2014, Jaud et al., 2012)

Sensor chl data availability

flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 chl AIS plots

  • Filtered to only 0 quality flags

2020 chl AIS plots

  • No QF filters on these plots
  • Is QF because of lack of calibration?

2019 Chl avg

  • Smooth the chl time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime chl5min chl_ma01 chl_ma03 chl_ma04 chl_ma04u chl_ma06 chl_ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 5.83 6.245 6.285 6.290 6.312174 6.350 6.610
2019-07-26 16:22:11 2019-07-26 16:20:00 6.70 6.175 6.285 6.310 6.327500 6.425 6.610
2019-07-26 22:09:25 2019-07-26 22:10:00 7.47 7.665 7.735 7.695 7.881042 7.660 7.650
2019-07-27 15:27:04 2019-07-27 15:25:00 8.16 8.195 8.380 8.400 8.477083 8.455 8.655
2019-07-30 15:34:20 2019-07-30 15:35:00 18.12 17.850 17.460 17.460 17.539130 17.860 19.670

2020 Chl avg

  • Smooth the chl time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble
  • Using median for 2020 because of QA issues? (or could remove outliers before calculating running median)

Get the moving average value closet to flight time

collectDateTime datetime chl5min chl_ma01 chl_ma03 chl_ma04 chl_ma04u chl_ma06 chl_ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 2.54 2.320 2.48 2.545 2.638333 2.715 3.150
2020-06-24 16:26:15 2020-06-24 16:25:00 2.74 2.565 2.56 2.565 2.581875 2.680 3.165
2020-06-24 16:18:44 2020-06-24 16:20:00 2.54 2.565 2.56 2.585 2.587917 2.680 3.175
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

Chl AIS vs AOS

  • use chla_df date and times with the aop/ais matching function to determine which sampling dates have sensor data from same day around time of sampling.

already sort of done for suna data… follow this pattern

  • Get chl sensor data from all AOS sampling dates (use ais with buffer 2 fucntion?)
  • show what is available
  • calculate rolling averages of chl time series
  • get rolling average values for time closest to sampling using which min on difftime
  • little overlap because of seasonal removal of buoys
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-11 13:53:00 TRUE TRUE TRUE TRUE
2019-09-10 13:37:00 TRUE TRUE TRUE TRUE
2020-06-15 15:09:00 TRUE TRUE TRUE TRUE
2020-09-29 13:21:00 FALSE FALSE FALSE FALSE
2021-04-20 14:40:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-11 13:53:00 TRUE TRUE TRUE TRUE
2019-09-10 13:37:00 TRUE TRUE TRUE TRUE
2020-06-15 15:09:00 TRUE TRUE TRUE TRUE

collect_date pheophytin chla chl_tot
2019-07-11 3.01 2.31 5.32
2019-09-10 3.08 2.79 5.87
2020-06-15 4.20 3.45 7.65
2020-09-29 72.40 124.44 196.84
2021-04-20 21.38 20.03 41.41

Compare AOS and AIS

DOM

AOS Sampling

  • From processed Surface water chemistry files on Box

EPA Method description:

  • The UVA procedure requires that the sample be passed through a 0.45 um filter and transferred to quartz cell. It is then placed in a spectrophotometer to measure the UV absorbance at 254 nm and reported in cm -1.

  • The SUVA procedure requires both the DOC and UVA measurement. The SUVA is then calculated by dividing the UV absorbance of the sample (in cm -1) by the DOC of the sample (in mg/L) and then multiplying by 100 cm/M. SUVA is reported in units of L/mg-M. The formula for the SUVA may be found in Section 12.2.

Ignoring data before 2016

## Warning: Removed 2 rows containing missing values (geom_point).

Correct older UV Absorbance wavelengths from 250 to 254, assuming all 250 nm should be 254.

swchem_site_df <- swchem_site_df %>%
  dplyr::mutate(analyte = dplyr::case_when(analyte == 'UV Absorbance (250 nm)' ~
                                             'UV Absorbance (254 nm)', TRUE ~ analyte))
## Warning: Removed 2 rows containing missing values (geom_point).

How much does SUVA 254/280 ratio change?

How much does organic carbon dissolved proportion change?

Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates

## Warning: Removed 11 rows containing missing values (geom_point).

Match DOM AOS and AOP data

Need to do this separately for each analyte because they arent always all reported.

DOC x flightdates

These are the closest DOC values (mg/L) for each flight date

  • A list of values means there are duplicate samples (multiple sampleIDs) from same date.
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2020-06-09 2020-08-04 15 41 15 TRUE TRUE
2020-06-26 2020-06-09 2020-08-04 17 39 17 TRUE TRUE
2020-07-02 2020-06-09 2020-08-04 23 33 23 TRUE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-06-09 15
2020-06-26 2020-06-09 17
2020-07-02 2020-06-09 23
aos_match flightdate days DOC
2017-06-08 2017-06-21 13 29.97
2017-07-06 2017-06-26 10 31.68, 31.81, 32.11
2019-08-06 2019-07-26 11 35.01
2019-08-06 2019-07-27 10 35.01
2019-08-06 2019-07-30 7 35.01
2020-06-09 2020-06-24 15 26.72
2020-06-09 2020-06-26 17 26.72
2020-06-09 2020-07-02 23 26.72

UV abs x flightdates

These are the closest UV abs values (per 1 cm) for each flight date

  • A list of values means there are duplicate samples (multiple sampleIDs) from same date.
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2016-10-19 NA 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2020-08-04 203 41 41 FALSE TRUE
2020-06-26 2019-12-04 2020-08-04 205 39 39 FALSE TRUE
2020-07-02 2019-12-04 2020-08-04 211 33 33 FALSE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-08-04 41
2020-06-26 2020-08-04 39
2020-07-02 2020-08-04 33
aos_match flightdate days UV Absorbance (254 nm)
2017-06-08 2017-06-21 13 0.5578
2017-07-06 2017-06-26 10 0.5566, 0.5710, 0.5653
2019-08-06 2019-07-26 11 0.7538
2019-08-06 2019-07-27 10 0.7538
2019-08-06 2019-07-30 7 0.7538
2020-08-04 2020-06-24 41 0.5646
2020-08-04 2020-06-26 39 0.5646
2020-08-04 2020-07-02 33 0.5646
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2016-10-19 NA 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2020-08-04 203 41 41 FALSE TRUE
2020-06-26 2019-12-04 2020-08-04 205 39 39 FALSE TRUE
2020-07-02 2019-12-04 2020-08-04 211 33 33 FALSE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-08-04 41
2020-06-26 2020-08-04 39
2020-07-02 2020-08-04 33
aos_match flightdate days UV Absorbance (280 nm)
2017-06-08 2017-06-21 13 0.2876
2017-07-06 2017-06-26 10 0.2887, 0.2961, 0.2950
2019-08-06 2019-07-26 11 0.3852
2019-08-06 2019-07-27 10 0.3852
2019-08-06 2019-07-30 7 0.3852
2020-08-04 2020-06-24 41 0.3118
2020-08-04 2020-06-26 39 0.3118
2020-08-04 2020-07-02 33 0.3118

TOC x flightdates

  • TOC is reported in milligramsPerLiter
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2020-06-09 2020-06-30 15 6 6 TRUE TRUE
2020-06-26 2020-06-09 2020-06-30 17 4 4 TRUE TRUE
2020-07-02 2020-06-30 2020-08-04 2 33 2 TRUE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-06-30 6
2020-06-26 2020-06-30 4
2020-07-02 2020-06-30 2
aos_match flightdate days TOC
2017-06-08 2017-06-21 13 34.12
2017-07-06 2017-06-26 10 37.87, 38.11, 38.82
2019-08-06 2019-07-26 11 36.05
2019-08-06 2019-07-27 10 36.05
2019-08-06 2019-07-30 7 36.05
2020-06-30 2020-06-24 6 26.05, 26.49, 26.18
2020-06-30 2020-06-26 4 26.05, 26.49, 26.18
2020-06-30 2020-07-02 2 26.05, 26.49, 26.18

Fe x flightdates

  • Fe is reported in milligramsPerLiter
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2021-05-04 203 314 203 FALSE FALSE
2020-06-26 2019-12-04 2021-05-04 205 312 205 FALSE FALSE
2020-07-02 2019-12-04 2021-05-04 211 306 211 FALSE FALSE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
aos_match flightdate days Fe
2017-06-08 2017-06-21 13 5e-04
2017-07-06 2017-06-26 10 5e-04, 5e-04, 5e-04
2019-08-06 2019-07-26 11 0.01
2019-08-06 2019-07-27 10 0.01
2019-08-06 2019-07-30 7 0.01

AIS Sensor

The EXO fDOM sensor is a fluorometer with a single excitation/emission pair (365nm/480nm) used to detect the fluorescent fraction of the chromophoric DOM when exposed to near‐UV light. Because of the impacts of temperature and water column absorbance (from a combination of dissolved and particulate compounds) on these readings corrections must be applied to the calibrated data.

  • availability of processed fDOM time series within different time windows
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 fDOM plots

## Warning: Removed 3170 rows containing missing values (geom_point).

## Warning: Removed 21 rows containing missing values (geom_point).

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

2020 fDOM plots

  • Not filtered on QFs
  • QF because of lack of calibration?
## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).
## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 rows containing missing values (geom_point).

2019 fDOM avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime fdom5min ma01 ma03 ma04 ma04u ma06 ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 109.76 109.620 109.620 109.630 109.6151 109.630 109.63
2019-07-26 16:22:11 2019-07-26 16:20:00 109.22 109.260 109.260 109.260 109.2767 109.260 109.26
2019-07-26 22:09:25 2019-07-26 22:10:00 108.23 108.075 108.065 108.065 108.0837 108.070 108.09
2019-07-27 15:27:04 2019-07-27 15:25:00 105.65 105.760 105.745 105.745 105.7490 105.745 105.80
2019-07-30 15:34:20 2019-07-30 15:35:00 100.23 100.380 100.350 100.310 100.2890 100.340 100.35

2020 fDOM avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime fdom5min ma01 ma03 ma04 ma04u ma06 ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 69.59 69.625 69.700 69.730 69.53479 69.865 70.200
2020-06-24 16:26:15 2020-06-24 16:25:00 70.02 69.855 69.880 69.880 69.94437 69.850 70.105
2020-06-24 16:18:44 2020-06-24 16:20:00 69.73 69.945 69.955 69.955 69.96188 69.850 70.105
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

fDOM vs DOC (TO DO)

  • use swchem_site_df in matching aop/ais function

TO DO!!!!

Suspended Matter

AOS Sampling

  • From processed Surface water chemistry files on Box
  • TDS passes through .45 micron filter
  • TSS is from weight of dried mass that doesnt pass through filter

Data from all locations at site

## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?

sm_site_df <- sm_site_df %>%
  dplyr::filter(namedLocation %in% my_loc)
## Warning: Transformation introduced infinite values in continuous y-axis

## Warning: Transformation introduced infinite values in continuous y-axis

How much does TDS/TSS change?

  • Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates
  • Dissolved passes through .45 micron filter, suspended does not
## Warning: Removed 1 row(s) containing missing values (geom_path).
## Warning: Removed 2 rows containing missing values (geom_point).

Match sediment aos to aop dates

source('R/match-aop-aos.R')

sm_match_list <- match_aop_aos(mysite_dates, sm_site_df, 'collect_date')

knitr::kable(sm_match_list$dates)
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2021-05-04 203 314 203 FALSE FALSE
2020-06-26 2019-12-04 2021-05-04 205 312 205 FALSE FALSE
2020-07-02 2019-12-04 2021-05-04 211 306 211 FALSE FALSE 
knitr::kable(sm_match_list$matches)
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7

TSS x flightdate

These are the closest in time ground-based measurements of TDS (mg/L) and TSS (mg/L)

  • List of values means there are duplicate samples (multiple sampleIDs) from same date.
aos_match flightdate days TDS TSS TSS - Dry Mass
2017-06-08 2017-06-21 13 910.1 32 800
2017-07-06 2017-06-26 10 885.62, 887.64, 891.23 10, 9, 9 900, 900, 1000
2019-08-06 2019-07-26 11 744.6 10 500
2019-08-06 2019-07-27 10 744.6 10 500
2019-08-06 2019-07-30 7 744.6 10 500

AIS Sensor

The EXO turbidity sensor employs a near‐IR light source (~780 ‐ 900 nm) and detects scattering at 90 degrees of the incident beam.

  • processed turbidity water quality time series on box
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 Turbidity plots

  • filtered to no quality flags

2020 Turbidity plots

  • Not filtered for quality flags
  • QF because of lack of calibration?
## Warning: Removed 8537 row(s) containing missing values (geom_path).
## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2438 row(s) containing missing values (geom_path).
## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).
## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 row(s) containing missing values (geom_path).
## Warning: Removed 4737 rows containing missing values (geom_point).

2019 turb avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime turb5min ma01 ma03 ma04 ma04u ma06 ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 2.09 2.170 2.170 2.175 2.228913 2.180 2.240
2019-07-26 16:22:11 2019-07-26 16:20:00 2.17 2.205 2.230 2.240 2.326667 2.245 2.290
2019-07-26 22:09:25 2019-07-26 22:10:00 3.98 3.185 3.140 3.125 3.129167 3.080 2.925
2019-07-27 15:27:04 2019-07-27 15:25:00 3.62 2.985 2.930 2.910 2.942917 2.790 2.600
2019-07-30 15:34:20 2019-07-30 15:35:00 2.75 2.770 2.795 2.820 2.921591 2.820 2.820

2020 turb avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime turb5min ma01 ma03 ma04 ma04u ma06 ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 1.91 1.865 1.820 1.865 3.284375 1.845 1.9
2020-06-24 16:26:15 2020-06-24 16:25:00 1.51 1.795 1.875 1.840 2.687708 1.875 1.9
2020-06-24 16:18:44 2020-06-24 16:20:00 1.93 1.795 1.875 1.815 2.677292 1.875 1.9
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

Turbidity vs TDS/ TSS (TODO)

  • use sm_site_df in matching function to get sensor aos overlap

TO DO!!!!

SUNA (move up later)

  • nitrate sensor 15 minute interval time series
  • value is from a burst of readings after lamp warms up
  • error bars on these plots are std error of mean, not expanded uncertainty like for other sensors
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE TRUE
2020-07-02 17:14:10 TRUE TRUE TRUE TRUE
flightline_datetime check_30day check_10day check_1day check_12hr
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE TRUE
2020-07-02 17:14:10 TRUE TRUE TRUE TRUE

Helper table for manually pulling SUNA data (AOP dates):

  • Helper table for pulling L0 SUNA for flight dates with SUNA data.
  • Interval is 3 days before and after to facilitate downloads.
  • Connect to VPN and use DQ Blizzard. View Level 0 data, extract L0 sub-stream
  • stream 006 is Absorbance at 254
  • stream 007 is Absorbance at 350
flightline_datetime check_1day interval3_start interval3_end
2019-07-26 15:20:23 TRUE 2019-07-23 2019-07-29
2019-07-26 16:22:11 TRUE 2019-07-23 2019-07-29
2019-07-26 22:09:25 TRUE 2019-07-23 2019-07-29
2019-07-27 15:27:04 TRUE 2019-07-24 2019-07-30
2019-07-30 15:34:20 TRUE 2019-07-27 2019-08-02
2020-06-24 17:36:27 TRUE 2020-06-21 2020-06-27
2020-06-24 16:26:15 TRUE 2020-06-21 2020-06-27
2020-06-24 16:18:44 TRUE 2020-06-21 2020-06-27
2020-06-26 19:28:06 TRUE 2020-06-23 2020-06-29
2020-07-02 17:14:10 TRUE 2020-06-29 2020-07-05
  • Those are the availabilities for the SUNA data at flight times.
  • Save 1 file for each year eg NEON-L0-SUNA/AOP-dates/{mysite}/{mysite}-103-2020.csv
  • Process raw suna data with process-L0-SUNA in processing project

2019 abs plots

## Rows: 803
## Columns: 9
## Delimiter: ","
## dbl  [8]: mean_abs254, sd_abs254, n_abs254, se_abs254, mean_abs350, sd_abs350, n_abs350,...
## dttm [1]: burst_id
## 
## Use `spec()` to retrieve the guessed column specification
## Pass a specification to the `col_types` argument to quiet this message

2019 abs avg

collectDateTime datetime mean_abs254 abs254_ma01 abs254_ma03 abs254_ma04 abs254_ma04u abs254_ma06 abs254_ma12
2019-07-26 15:20:23 2019-07-26 15:15:00 0.7239778 0.7223167 0.7223167 0.7223500 0.7245560 0.7223500 0.7219900
2019-07-26 16:22:11 2019-07-26 16:15:00 0.7213222 0.7213500 0.7219522 0.7219522 0.7234967 0.7219522 0.7217889
2019-07-26 22:09:25 2019-07-26 22:15:00 0.7212750 0.7201167 0.7181789 0.7181789 0.7174843 0.7181789 0.7181800
2019-07-27 15:27:04 2019-07-27 15:30:00 0.6946000 0.6943750 0.6928211 0.6925200 0.6904142 0.6928211 0.6931222
2019-07-30 15:34:20 2019-07-30 15:30:00 0.6522333 0.6523500 0.6522917 0.6531444 0.6535957 0.6559706 0.6577222
collectDateTime datetime mean_abs350 abs350_ma01 abs350_ma03 abs350_ma04 abs350_ma04u abs350_ma06 abs350_ma12
2019-07-26 15:20:23 2019-07-26 15:15:00 0.1960778 0.1944944 0.1944944 0.1944944 0.1966453 0.1944944 0.1938222
2019-07-26 16:22:11 2019-07-26 16:15:00 0.1936000 0.1936050 0.1936050 0.1936050 0.1953644 0.1936050 0.1930600
2019-07-26 22:09:25 2019-07-26 22:15:00 0.1911556 0.1904989 0.1902956 0.1902956 0.1886990 0.1902956 0.1901800
2019-07-27 15:27:04 2019-07-27 15:30:00 0.1660222 0.1659350 0.1653950 0.1653000 0.1635622 0.1653950 0.1654900
2019-07-30 15:34:20 2019-07-30 15:30:00 0.1245444 0.1245444 0.1242889 0.1257833 0.1268550 0.1278417 0.1314700

2020 abs plots

## Rows: 755
## Columns: 9
## Delimiter: ","
## dbl  [8]: mean_abs254, sd_abs254, n_abs254, se_abs254, mean_abs350, sd_abs350, n_abs350,...
## dttm [1]: burst_id
## 
## Use `spec()` to retrieve the guessed column specification
## Pass a specification to the `col_types` argument to quiet this message

2020 abs avg

collectDateTime datetime mean_abs254 abs254_ma01 abs254_ma03 abs254_ma04 abs254_ma04u abs254_ma06 abs254_ma12
2020-06-24 17:36:27 2020-06-24 17:30:00 0.4602 0.4606 0.4606 0.4607 0.4609 0.4609 0.4611
2020-06-24 16:26:15 2020-06-24 16:30:00 0.4600 0.4600 0.4602 0.4604 0.4608 0.4609 0.4611
2020-06-24 16:18:44 2020-06-24 16:15:00 0.4598 0.4600 0.4602 0.4604 0.4609 0.4608 0.4612
2020-06-26 19:28:06 2020-06-26 19:30:00 0.4726 0.4727 0.4727 0.4727 0.4733 0.4727 0.4732
collectDateTime datetime mean_abs350 abs350_ma01 abs350_ma03 abs350_ma04 abs350_ma04u abs350_ma06 abs350_ma12
2020-06-24 17:36:27 2020-06-24 17:30:00 0.0449 0.0450 0.0452 0.0452 0.0455 0.0457 0.0456
2020-06-24 16:26:15 2020-06-24 16:30:00 0.0449 0.0447 0.0448 0.0450 0.0453 0.0455 0.0457
2020-06-24 16:18:44 2020-06-24 16:15:00 0.0443 0.0447 0.0447 0.0450 0.0453 0.0454 0.0458
2020-06-26 19:28:06 2020-06-26 19:30:00 0.0545 0.0548 0.0548 0.0548 0.0550 0.0548 0.0554

x2019 SUNA plots 2019

  • filtered to no Quality Flags

x2020 SUNA plots 2020

  • Not filtered by QFs
  • QF because of lack of calibration?

SUNA vs AOS absorbs

  • Use surface water chemistry data to see date and times of ground sample collection for UV absorbance data

This is the availability of SUNA data for AOS sampling dates

aos_datetime check_3day check_1day check_6hr check_1hr
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 FALSE FALSE FALSE FALSE
2018-08-07 16:16:00 FALSE FALSE FALSE FALSE
2018-09-04 15:50:00 FALSE FALSE FALSE FALSE
2018-10-16 16:03:00 FALSE FALSE FALSE FALSE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 FALSE FALSE FALSE FALSE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2020-08-04 15:07:00 FALSE FALSE FALSE FALSE
2020-09-02 13:27:00 FALSE FALSE FALSE FALSE
2021-01-07 16:08:00 FALSE FALSE FALSE FALSE
2021-02-02 17:15:00 FALSE FALSE FALSE FALSE
2021-03-02 18:05:00 FALSE FALSE FALSE FALSE
2021-04-07 15:45:00 TRUE FALSE FALSE FALSE
2021-05-04 15:22:00 TRUE TRUE TRUE TRUE
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2021-04-07 15:45:00 TRUE FALSE FALSE FALSE
2021-05-04 15:22:00 TRUE TRUE TRUE TRUE

Helper table for manually pulling SUNA data (AOS dates):

aos_datetime check_1day interval3_start interval3_end
2019-07-01 14:53:00 TRUE 2019-06-28 2019-07-04
2019-08-06 15:55:00 TRUE 2019-08-03 2019-08-09
2019-09-04 16:00:00 TRUE 2019-09-01 2019-09-07
2019-10-16 17:05:00 TRUE 2019-10-13 2019-10-19
2021-04-07 15:45:00 FALSE 2021-04-04 2021-04-10
2021-05-04 15:22:00 TRUE 2021-05-01 2021-05-07

SUNA level 0 absorbances

  • processed SUNA absorbance data are in box folder NEON-processed/suna-L0-timeseries
  • how do absorbances from nitrate sensor compare to AOS samples?

Regression for sensor data within the same day

## `geom_smooth()` using formula 'y ~ x'